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(54) Information processing apparatus and method 



(57) In an apparatus such as a digital camera or 
color facsimile apparatus using JPEG coding, since an 
input image is coded/compressed by hardware, a wa- 
termark embedding process before compression/cod- 



ing causes a great increase in cost. For this reason, cod- 
ed data or quantized data is input, and watermark infor- 
mation is embedded in the input data by using the table 
used to code or quantize the input data. 
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Description 

BACKGROUND OF THE INVENTION 

5 FIELD OF THE INVENTION 

[0001] The present invention relates to an information processing apparatus, information processing method, and 
recording medium and, more specifically, to an information processing apparatus and method which embed specific 
information such as watermark information in coded or quantized data, and a recording medium. 

10 

DESCRIPTION OF THE RELATED ART 

[0002] With the recent remarkable advances in computers and networks, so-called multimedia information such as 
character information, image information, and speech information is handled by computers through networks. Since 

15 image and speech data are relatively large in data amount, the data amount is reduced by coding/compressing the 
image and speech data. If image data is compressed, more image information can be transmitted through a network 
at high speed. As an image compression technique, commonly called JPEG (Joint Photographic Expert Groups) for 
compressing multilevel still images according to ITU-T recommendation T81 is widely used. To attain high-efficiency 
compression, standardization of JPEG 2000 has been examined. 

20 [0003] JPEG is basically a coding scheme that uses the discrete cosine transformation (DCT). As a JPEG 2000 
scheme, a scheme that uses the discrete wavelet transformation (DWT) is promising. As shown in Fig. 1A, each of 
JPEG and JPEG 2000 coding apparatuses is generally comprised of a frequency converter 701 for calculating the 
discrete cosine transform and discrete wavelet transform, quantizer 702, entropy coder 703, and the like. As shown in 
Fig. 1 B, each of JPEG and JPEG 2000 decompressing apparatuses is comprised of an entropy decoder 704, inverse 

25 quantizer 705, and inverse frequency converter 706. 

[0004] Image and speech data used in a computer or network are digital data, and hence can be easily duplicated. 
This duplication causes no deterioration in data quality. For this reason, to protect the copyright on these pieces of 
multimedia information, copyright information is embedded as watermark information in image and speech data. By 
extracting the watermark information from the multimedia, the copyright information can be obtained, thus preventing 

30 unauthorized duplication of the data. 

[0005] Examples of watermark embedding methods are a scheme adopted by Nippon Telephone Telegraph (NTT), 
which uses the discrete cosine transform, "Watermarking Scheme in Frequency Domain for Protecting Copyright of 
Digital Image" (Nakamura, Ogawa, and Takashima, SCIS'97-26A, January 1997), scheme adopted by Defense Acad- 
emy, which uses the discrete Fourier transform, "Watermark Signature Method for Images based on PN Sequences' 

35 (Onishi, Oka, and Matsui, SCIS'97-26B, January 1997), and scheme adopted by Mitsubishi Corp. and Kyushu Univer- 
sity, "Empirical Examination about Safety and Reliability of Watermarking Technique using Wavelet transform" (Ishi- 
zuka, Sakai, and Sakurai, SCIS'97-26D, January 1997. 

[0006] As shown in Fig. 2A, such a watermark embedding apparatus is comprised of a frequency converter 801 for 
calculating the discrete cosine transform or discrete wavelet transform, watermark embedding device 802 for quantizing 
40 the frequency components in accordance with information to be embedded, and inverse frequency converter 803. A 
watermark extracting unit is comprised of a frequency converter 804 and watermark extracting device 805 for extracting 
the information embedded in quantized frequency components, as shown in Fig. 2B. 

[0007] When a watermark embedding process and compressing process are to be performed for image data, it is 
easy to compress the image data in which watermark information is embedded. It is, however, difficult to embed wa- 
45 termark information in compressed data. Since compressed data is entropy-coded data, it is difficult to directly embed 
watermark information in the coded data. 

[0008] Watermark information is embedded in compressed data in the following manner. As shown in Fig. 3, image 
data x compressed by components ranging from an image converter 902 to an entropy coder 904 is decompressed to 
be temporarily restored to image data by components ranging from an entropy decoder 905 to an image inverse con- 
50 verter 907. Watermark information is embedded in the image data by components ranging from an image converter 
908 to an image inverse converter 910. The resultant output is re-compressed by components ranging from an image 
converter 911 to an entropy coder 913. 

[0009] In contrast to this, the output obtained by a watermark embedding process is image data itself, and hence 
can be directly input to a compressing process. 
55 [0010] Various applications and apparatuses using coding schemes such as JPEG, e.g., digital cameras, color fac- 
simile apparatuses, still image transfer systems, and still image processing systems, have been put into practice. Many 
of these applications and apparatuses use hardware such as Applicatbn Specific Integrated Circuits (ASICs) to code 
input images first, i.e., compress input images. If, therefore, these applications and apparatuses are designed to embed 



3 



EP 1 001 604 A2 



watermark information before a compressing/coding process, a great increase in cost is inevitable. 
[0011] In addition, as sufficient studies have not been made to integrate a compressing/coding process and water- 
mark embedding process in JPEG 2000 under examination, the problem of difficulty in embedding watermark infor- 
mation is likely to arise as in the case with JPEG. 

SUMMARY OF THE INVENTION 

[0012] The present invention aims to address one or more of the above problems, and has as an object to provide 
an information processing apparatus and method which embed added information such as watermark information in 
coded or quantized data. 

[0013] In order to achieve the above object, according to a preferred embodiment of the present invention, there is 
provided an image processing apparatus comprising inputting means for inputting image data which has been coded 
in accordance with a coding table, and embedding means for embedding added information in the image data by 
modifying the input image data based on the coding table. 

[0014] In addition, there is provided an information processing apparatus comprising inputting means for inputting 
image data which has been quantized in accordance with a quantizing table, and embedding means for embedding 
added information in the image data by modifying the input image data based on the quantizing table. 
[001 5] It is another object of the present invention to provide an information processing apparatus and method which 
embed specific information such as watermark information in coded or quantized data. 

[0016] In order to achieve the above object, according to a preferred embodiment of the present invention, there is 
provided an information processing apparatus comprising decoding means for decoding and inversely quantizing coded 
and quantized data, embedding means for embedding added information in the data output by the decoding means, 
and re-coding means for quantizing and coding the data output by the embedding means. 

[001 7] In addition, there is provided an information processing method comprising the steps of decoding and inversely 
quantizing coded and quantized data, embedding added information in the data output in the decoding step, and re- 
quantizing and re-coding the data output in the embedding step. 

[0018] There is also provided an information processing apparatus comprising decoding means for decoding coded 
data, embedding means for embedding added information in the data output by the decoding means, and re-coding 
means for coding the data output by the embedding means. 

[0019] Furthermore, there is provided an information processing apparatus comprising decoding means for decoding 
and inversely quantizing coded and quantized data, and extracting means for extracting added information embedded 
in the data which is output by the decoding means. 

[0020] Moreover, there is provided an image processing apparatus comprising generating means for generating 
image data, compressing means for compressing the image data generated by the generating means, and embedding 
means for embedding added information in the image data compressed by the compressing means, wherein the com- 
pressing means and the embedding means can be executed in independently. 

[0021] Other features and advantages of the present invention will be apparent from the following description taken 
in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts 
throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0022] 

Fig. 1 A is a block diagram for briefly describing an image coding apparatus; 
Fig. 1 B is a block diagram for briefly describing an image decoding apparatus; 
Fig. 2A is a block diagram for briefly explaining a watermark embedding apparatus; 
Fig. 2B is a block diagram for briefly explaining a watermark extracting apparatus; 

Fig. 3 is a block diagram for briefly explaining an apparatus for embedding watermark information in compressed 
data; 

Fig. 4 is a view showing a quantizing table; 

Fig. 5 is a view fore explaining a zigzag scan; 

Fig. 6 is a view showing a Huffman table; 

Fig. 7 is a view for explaining the format of JPEG data; 

Figs. 8A and 8B are block diagrams showing the arrangement of an image processing apparatus according to the 
first embodiment of the present invention; 

Fig. 8C is a block diagram showing the arrangement of an image processing apparatus according to the second 
embodiment; 
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Figs. 9A to 9C are block diagrams each showing the arrangement of an image processing apparatus according 
to the fourth embodiment; 

Fig. 10 is a block diagram showing the arrangement of an image processing apparatus according to the fifth em- 
bodiment; 

5 Fig. 11 is a block diagram showing the arrangement of an image processing apparatus according to the sixth 

embodiment; 

Fig. 1 2A is a block diagram showing the arrangement of an image processing apparatus according to the seventh 
embodiment; 

Fig. 12B is a block diagram showing the second example of the image processing apparatus according to the 
io seventh embodiment; 

Fig. 1 3 is a block diagram showing the third example of the image processing apparatus according to the seventh 
embodiment; 

Fig. 14 is a block diagram showing the fourth example of the image processing apparatus according to the fifth 
embodiment; 

J5 Fig. 15 is a block diagram showing the arrangement of an image processing apparatus according to the eighth 

embodiment; and 

Fig. 16 is a block diagram showing the second example of the image processing apparatus according to the eighth 
embodiment. 



20 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



[0023] An image processing apparatus which is an information processing apparatus according to each embodiment 
of the present invention will be described in detail below with reference to the accompanying drawings. The present 
invention relates to an information processing apparatus and method which protect rights such as a copyright, prevent 
25 unauthorized duplication and tampering of multimedia information such as image information, video information, and 
sound information including speech information, and record various information by compressing digital image data and 
embedding watermark information in the data. 

[0024] In the following embodiments, the process of embedding watermark information in JPEG -coded compressed 
data and the process of extracting the embedded watermark information will be described. A JPEG compressing proc- 
30 ess, JPEG decompressing process, and watermark information embedding process will be briefly described first. 



(JPEG Coding] 

[0025] A JPEG compressing process will be briefly described with reference to Fig. 1A showing a JPEG coding 
35 process procedure. 

[0026] First of all, an image converter 701 divides still image data to be coded into 8 x 8-pixel blocks, and performs 
the DCT (Discrete Cosine Transformation) in units of blocks. Each 8 x 8-pixel block having undergone the DCT will be 
referred as a "DCT coefficient block"; a coefficient contained in a DCT coefficient block, as a "DCT coefficient"; and a 
set of DCT coefficient blocks corresponding to one image, as a "DCT coefficient block group". 

40 [0027] A quantizer 702 then quantizes a DCT coefficient block group by using an arbitrary quantizing table. A DCT 
coefficient block that is quantized will be referred to as a "quantized DCT coefficient block"; and a set of quantized DCT 
coefficient blocks corresponding to one image, as a "quantized DCT coefficient block group". 
[0028] An entropy coder 703 Huffman-codes a quantized DCT coefficient block group. This Huffman-coded data 
becomes JPEG data. The Huffman table used in this case may be a table prepared in advance or table generated for 

45 each image compressing process. The JPEG data includes the Huffman-coded data, quantizing table used for com- 
pression, and Huffman table. 

[0029] A quantized DCT coefficient block is obtained by dividing a DCT coefficient block by the value determined by 
a quantizing table. For example, a quantizing table for color difference components is provided for a 8 x 8 DCT coefficient 
block, as shown in Fig. 4. This quantizing table is designed to assign more bits to low-frequency components and fewer 
50 bits to high-frequency components. If, therefore, a DCT coefficient is -36 and the corresponding value in the quantizing 
table is 18, the quantized DCT coefficient is -2. If the corresponding value in the quantizing table is 99, the quantized 
DCT coefficient is 0. 

[0030] The DC and AC coefficients of the quantized DCT coefficients contained in a quantized DCT coefficient block 
are separately Huffman-encoded. Each DC correction coefficient is transformed into a difference between itself and 
55 the DC coefficient of the immediately preceding quantized DCT coefficient block and coded by using a table for DC 
coefficients. The AC coefficients are rearranged in a zigzag scan order like the one shown in Fig. 5, and are coded by 
using a table having a combination of runs of successive zero coefficients (zero run lengths) and coefficient values 
other than zero. Fig. 6 shows an example of a Huffman table for AC coefficients. The AC coefficients are classified 
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into groups according to their values, and codes corresponding to the zero run lengths are assigned to the respective 
groups, using the table in Fig. 6. If, for example, an AC coefficient is -2, the coefficient is classified as a coefficient 
belonging to group 2. If the corresponding zero run length is 0, the code "01 " is assigned to the group. Note that if an 
added bit for discriminating the AC coefficient from the remaining coefficients in the group is "01", this AC coefficient 
is represented by "01 01". 

[Format of JPEG Data] 

[0031] Fig. 7 shows JPEG data, and more specifically, the format of the data compressed by the JPEG sequential 
scheme. According to the sequential scheme, in decoding operation, the original image is sequentially reconstructed 
from the upper side of the image. In contrast to this, a scheme of displaying unclear images as a whole at first and 
then gradually displaying clearer images is called the progressive scheme. These different schemes are selectively 
used depending on the Huffman coding method in the entropy coder 703 in Fig. 1 A. 

[0032] JPEG data is formatted by using 2-byte codes called markers which are unique in the data. The first SOI 
(Start of Image) marker represents the start of the JPEG data. The next DQT (Define Quantization Table) marker fields 
defines a quantizing table. The quantizing table used in compression is stored after the DOT marker 
[0033] The next SOFO (Start Of Frame 0) marker field is used for JPEG data compressed by the sequential scheme 
using the DCT Parameters in compression, e.g., the image size of the compressed image, sampling ratio, component 
count, and the identifier of the quantizing table for each component, are stored next to the SOFO marker. 
[0034] The next DHT (Define Huffman Table) marker field defines a Huffman table. The Huffman table used in com- 
pression is stored behind the DHT marker. The next SOS (Start Of Scan) marker field is used to store the Huffman 
codes of a coded image. The Huffman codes of an image coded after a header consisting of several bytes and con- 
taining information such as the identifier of the Huffman table for each component is added to the image. The last EOI 
(End of Image) marker indicates the end of the JPEG data. 

[0035] The order of the markers is not limited to the one shown in Fig. 7. Some JPEG data may include two or more 
identical markers or marker fields. 

[Decoding of JPEG Data] 

[0036] Fig. 1 B is a block diagram showing a decoding process procedure for the JPEG data compressed according 
to the procedure shown in Fig. t A. 

[0037] In decoding the JPEG data, an entropy decoder 704 decodes the Huffman codes into a quantized DCT co- 
efficient block group by using the Huffman table in the JPEG data. An inverse quantizer 705 inversely quantizes the 
quantized DCT coefficient block group into a DCT coefficient block group by using the quantizing table in the JPEG 
data. An image inverse converter 706 then computes the inverse DCT (IDCT) for the DCT coefficient block group to 
reconstruct a 8 x 8-pixel block, thus reconstructing an image. 
[0038] Decoded image data can be obtained from JPEG data by this procedure. 

[Watermark Embedding Process] 

[0039] A watermark embedding process will be described next with reference to Fig. 2A. The watermark embedding 
process described below is implemented by manipulating DCT coefficient blocks. The watermark embedding process 
is performed by the following procedure. 

[0040] First of all, an image converter 801 computes the DCT of an input image. A watermark embedding device 
802 then selects one or more DCT coefficient blocks (to be referred to as "embedding blocks 0 hereinafter) in which 
watermark information is embedded from the DCT coefficient block, and embeds one of the bits constituting the wa- 
termark information in each of the selected embedding blocks. Note that DCT coefficients in which bits are embedded 
are randomly selected from coefficients having relatively low frequencies, except for DC coefficients, in the embedding 
blocks. 

[0041] The watermark embedding process is completed by quantizing the embedding blocks. The size of a quanti- 
zation step in this case corresponds to the embedding strength. 

[0042] The following is an example of embedding the bit "0" or "V for embedding watermark information (to be 
referred to as a "embedding bit" hereinafter). A value q is obtained by quantizing s{u 0 v 0 } by: 

q = «s{u 0 v 0 }/h»h (1) 
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where s{u 0 v 0 ) is the value of a DCT coefficient, h is the embedding strength, and «x» is a maximum integer that 
does not exceed x. 

[0043] A value c that satisfies equation (2) or (3) is set as a DCT coefficient after the watermark embedding process. 
[0044] If the embedding bit is "0°, then 

c = q + h/4 (2) 



[0045] If the embedding bit is "1 ", then 

10 

c = q + 3h/4 (3) 
[0046] Finally, an image inverse converter 803 inversely converts the output form a watermark embedding device 802. 

75 

<First Embodiment 

[0047] Figs. 8A and 8B are block diagrams showing the arrangement of an image processing apparatus according 
to the first embodiment of the present invention. 

20 [0046] Referring to Fig. 8A, the data input from a memory 101 to a watermark embedding unit 102 is JPEG data 
having the data format shown in Fig. 7. The watermark embedding unit 102 performs the above watermark embedding 
process for the input JPEG data. Fig. 8B shows an arrangement for this watermark embedding process. 
[0049] In the watermark embedding process procedure described above, embedding blocks are selected from a 
DCT coefficient block group. Referring to Fig. 8B, however, a embedding coefficient selector 103 selects embedding 

25 blocks from the Huffman-coded data succeeding the SOS marker in Fig. 7. 

[0050] The embedding coefficient selector 1 03 also selects DCT coefficients in which embedding bits are to be em- 
bedded from coefficients having relatively low frequencies, except for the DC coefficients, in the embedding blocks. 
Since the DCT coefficients are arranged in the order determined by the zigzag scanning shown in Fig. 5, this selection 
has a unique order . 

30 [0051] As described above, the watermark embedding process is completed by quantizing the embedding blocks. 
This indicates that the original values of the DCT coefficients are changed. The original values of the DCT coefficients 
can be obtained by decoding the coded data by using a Huffman table and quantizing table. The DCT coefficients 
changed by the watermark embedding process are coded again by using the Huffman table and quantizing table. The 
watermark embedding process is therefore implemented by a code changing device 104 for changing input coded data 

35 by using the Huffman table behind the DHT marker and quantizing table next to the DQT marker in Fig. 7 as parameters. 
[0052] If, for example, the coded data corresponding to a selected DCT coefficient is "0101 ", a quantized value of - 
2 is obtained by using the Huffman table. If the value of the quantizing table which corresponds to this DCT coefficient 
is 18, the original value of the DCT coefficient is -36. Assume that the embedding bit "1 * is to be embedded in this DCT 
coefficient. In this case, if the embedding strength is represented by h = 18, c = -49 is obtained from equations (1) and 

40 (3). When the DCT coefficient is quantized by using the same quantizing table, a quantized DCT coefficient of -3 is 
obtained. In addition, the code "01 B is obtained by using the same Huffman table. Note that if the added bit for discrim- 
inating the AC coefficients in the group is "00", this coefficient is represented by "0100". That is, the code "0101" is 
rewritten into "0100" to realize watermark embedding. If, however, the code length changes upon watermark embed- 
ding, the associated data and the like next to the SOF marker must be rewritten. 

45 [0053] As described above, according to the first embodiment, watermark information is embedded in JPEG data 
by directly rewriting the JPEG data by using a Huffman table and quantizing table. 

[0054] Obviously, the processing in the first embodiment can be applied to various watermark embedding methods 
of embedding watermark information by manipulating DCT coefficients as well as the above watermark embedding 
method. 

50 

<Second Embodiment 

[0055] An image processing apparatus according to the second embodiment of the present invention will be described 
below. The same reference numerals as in the first embodiment denote the same parts in the second embodiment, 
55 and a detailed description thereof will be omitted. 

[0056] In the first embodiment, watermark information is embedded by manipulating the values of DCT coefficients 
of embedding blocks. The processing in the first embodiment therefore requires a Huffman table and quantizing table 
to reconstruct the DCT coefficients from JPEG data. 
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[0057] In the second embodiment, a method of embedding watermark information by manipulating the values of 
quantized DCT coefficients of quantized DCT coefficient blocks, i.e., a method of embedding watermark information 
in JPEG data without using any quantizing table, will be described. According to the second embodiment, since wa- 
termark information is embedded in quantized DCT coefficient blocks without reconstructing DCT coefficient blocks 
from JPEG data, only a Huffman table is required. 

[0058] In the method of embedding watermark information by manipulating the values of quantized DCT coefficients 
if the value of a quantized DCT coefficient is an even number, the embedding bit "0" is embedded, whereas if this value 
is an odd number, the embedding bit "1" is embedded. If, for example, the embedding bit '1 " is to be embedded in the 
code data 0101 ■, since the value of the quantized DCT coefficient corresponding to the code is the even number -2 
this value is changed to the nearest odd number -3. The code '0101' is written into '01 00", accordingly. When the 
embedding bit "0' is to be embedded in the code data "0101 ", the code data remains unchanged 
[0059] A watermark embedding process can also be implemented by a method of embedding bit "0" if the value of 
a quantized DCT coefficient belongs to an even-numbered group, and embedding the embedding bit "1 " if this value 
™ZT ,0 ^ odd - number9d 9 rou P Whs". sample, the embedding bit "1" is to be embedded in the code data 
0101 since the quantized DCT coefficient corresponding to the code belongs to group 2 (even-numbered group) 
™ deis 7 ttenintoacodebel °n9>ngtogroup1 or 3, which is the nearest odd-numbered group. When the embedding 
bit 0 is to be embedded in the code data "0101 the code data remains unchanged 

[0060] In addition, according to the watermark embedding method of the first embodiment, a watermark embeddinq 
process using only a Huffman table is implemented by setting the value q or «{u 0 V 0 }/h» in equation (1 ) as the value 
of a quantized DCT coefficient. 

[0061] Fig. 8C is a block diagram showing the arrangement of a watermark embedding unit 102 for embedding 
watermark information in JPEG data by the watermark embedding method using onhy a Huffman table. Note that data 
in which watermark information is embedded is JPEG data having, for example, a data format like the one shown in 

[0062] As in the first embodiment, a embedding coefficient selector 203 selects code data in which embeddinq blocks 
and embedding bits are to be embedded from the Huffman-coded data succeeding the SOS marker in Fig 7 
[0063] A code changing device 204 for implementing the above watermark embedding method directly rewrites the 
selected coded data by using only the Huffman table next to the DHT marker in Fig. 7, thereby embedding the embed- 
ding bits. If, however, the code length changes upon watermark embedding, the associated data and the like behind 
the SOF marker must be rewritten. 

[0064] Obviously, the processing in the second embodiment can be applied to various watermark embedding meth- 
method 9 wa,ermark ^"nation by operating DCT coefficients as well as the above watermark embedding 



<Third Embodiment 



[0065] An image processing apparatus according to the third embodiment of the present invention will be described 
wT , T ^ e J same refer *nce numerals as in the first embodiment denote the same parts in the third embodiment and 
a detailed description thereof will be omitted. 

[0066] In the first and second embodiments, the methods of embedding watermark information in coded/compressed 
data using the discrete cosine transformation, e.g., JPEG data, have been described. In the third embodiment a dis- 
crete cosine transformation device is replaced with various image converters such as a discrete wavelet transformation 
device, discrete Fourier transformation device, and predictive coder. The present invention can be applied to various 
image converters owing to the following characteristic feature of the present invention. According to the present inven- 
tion, entropy-coded data including Huffman codes and data compressed by quantization or the like are decoded up to 
an intermediate stage by looking up the tables used for the coding and compressing processes essentially without any 
dependence on image conversion such as the discrete cosine transformation, and watermark information is embedded 
in the data at the intermediate stage. 

[0067] In addition, the watermark embedding method of the present invention can be applied to any methods of 
operating the data obtained by decoding compressed data or data decoded up to an intermediate stage The format 
ofcompressed data is not limited to that of JPEG data shown in Fig. 7. Watermark information can be embedded in 
coded/compressed data having any formats as long as information indicating the location of the compressed data or 
the table used for compression can be uniquely specified. 

[0068] JPEG 2000 for which standardization is currently underway has an arrangement like the one shown in Fiq 
1 A^ In this scheme, a discrete wavelet transformation device is promising as an image converter, and an entropy coding 
scheme including a Huffman scheme is promising as an coding scheme. Obviously, therefore, for the watermark em- 
^S"££! f u° f mani P ula,in 9 discre,e coefficients, a watermark embedding apparatus corresponding to 

JPEG 2000 can be realized by the arrangement of the first and second embodiments shown in Figs 8A to 8C 



8 



EP 1 001 604 A2 



[0069] In each embodiment described above, a watermark embedding process is targeted on image data. As is 
apparent, however, the targets in these embodiments are not limited to image data, and the embodiments can be 
applied to multimedia data such as video data and sound data including speech data. 

5 <Fourth Embodiment 

[0070] An image processing apparatus according to the fourth embodiment of the present invention will be described 
below. The same reference numerals as in the first embodiment denote the same parts in the fourth embodiment, and 
a detailed description thereof will be omitted. 
10 [0071] In the fourth embodiment, a method of extracting watermark information from compressed data in which the 
watermark information is embedded will be described. Figs. 9A to 9C are block diagrams each showing the arrangement 
of an image processing apparatus according to the fourth embodiment. 

[0072] The process of extracting watermark information embedded in data by the method of the first embodiment 
will be described first. 

is [0073] Referring to Fig. 9A, the data input from a memory 301 to a watermark extracting unit 302 is JPEG data 
having, for example, a data format like the one shown in Fig. 7. The watermark extracting unit 302 extracts watermark 
information from the input JPEG data. Fig. 9B shows an arrangement for this watermark extracting process. 
[0074] First of all, each extraction block from which watermark information is to be extracted is selected by an ex- 
traction coefficient selector 303 designed to perform processing similar to that performed by the embedding coefficient 

20 selector 103 in Fig. 8B. 

[0075] When watermark information is to be extracted from the selected extraction block, a code extracting device 
304 transforms the JPEG data of the extraction block into a DCT coefficient by using a Huffman table and quantizing 
table, and determines whether the value corresponds to "1 B or "0\ For example, the code data ■0100° is decoded into 
the quantized DCT coefficient -3 t and the corresponding value in the quantizing table is decoded into the DCT coefficient 

25 -48. In addition, the embedding bit "1 B indicating watermark information is extracted from the decoded DCT coefficient 
according to equations (1 ) and (3). 

[0076] The method of extracting embedded watermark information according to the second embodiment will be 
described next with reference to Fig. 9C. 

[0077] First of all, each extraction block from which watermark information is to be extracted is selected by an ex- 
30 traction coefficient selector 403 designed to perform processing similar to that performed by the embedding coefficient 
selector 203 in Fig. 8C. A code extracting device 404 then determines whether the watermark information embedded 
in a quantized DCT coefficient of the extraction block is "1 " or "0". For example, since it is found out from a Huffman 
table that the value of the quantized DCT coefficient corresponding to the code data "0100" is an odd number, '1 ■ is 
extracted as watermark information. 
35 [0078] In this manner, watermark information is directly extracted from JPEG data by using a quantizing table and/ 
or Huffman table. 

[0079] Obviously, the watermark information extracting process in the fourth embodiment corresponds to the water- 
mark embedding process using various image converters in the third embodiment as well as the watermark embedding 
process using the discrete cosine transformation shown in the first and second embodiments. 

40 

<Fifth Embodiment 

[0080] Fig. 10 is a block diagram showing the arrangement of an image processing apparatus according to the fifth 
embodiment. The flow of processing will be briefly described first. 
45 [0081] Referring to Fig. 10, input image data x is multilevel image data having a predetermined number of bits per 
pixel. A discrete cosine transformation device 1102 performs a predetermined transformation process for the input 
multilevel image data to decompose the data into predetermined frequency components. The pixels of the input image 
data are segmented into separate blocks, and the discrete cosine transform is computed in units of blocks. The discrete 
cosine transform can be computed by 

so 

xi(u,v) = j?C(u)C(v) X u I vx j( m,n)cos < 2m ^) u « cos^j^ (4) 

for 

55 

C(p) = 1/\'2 when p = 0, and 
C(p) = 1 when P *0 



9 



EP 1 001 604 A2 



[0082] Since the image data of a natural image contains many low-frequency signals, the signals can be offset to 
the low-frequency region by the discrete cosine transformation. The transform coefficients (to be referred to as "DCT 
coefficients" hereinafter) output from the discrete cosine transformation device 1102 are input to a quantizer 1103. 
[0083] As described above, since the image data of a natural image contains many low-frequency signals, the image 
data can be efficiently compressed by assigning more bits to the low-frequency components of the image data and 
fewer bits to the high-frequency components by quantization. 

[0084] The quantized image data is input to an entropy coder 1 1 04 for executing Huffman coding or the like, in which 
a long code word is assigned to each information with a low occurrence frequency and a short code word is assigned 
to each information with a high occurrence frequency As a consequence, the average code word length decreases. 
The compressed image data having undergone entropy coding is input to a memory 1105 such as a magnetic storage 
unit. 

[0085] The process of embedding watermark information in the compressed image data stored in the memory 1105 
will be described below. 

[0086] The entropy-coded compressed image data read out from the memory 1 1 05 is decoded by an entropy decoder 
1 106 corresponding to the entropy coder 1104 used for the compressing process to obtain quantized data. This quan- 
tized data is inversely quantized by an inverse quantizer 1109 corresponding to the quantizer 1103 used for the com- 
pressing process to be restored to the DCT coefficients. A watermark embedding device 1 1 07 then embeds watermark 
information in the DCT coefficients. That is, the watermark embedding device 11 07 outputs the DCT coefficients having 
watermark information embedded therein by manipulating the DCT coefficients. 

[0087] The DCT coefficients in which the watermark information is embedded are compressed again by a quantizer 
1110 and entropy coder 1108. The output from the entropy coder 1108 is stored in the memory 1105 
[0088] The following scheme is available as a watermark embedding scheme using the discrete cosine transforma- 
tion. 

[0089] The DCT coefficients obtained by breaking up an input image into 8 x 8-pixel square blocks and performing 
the discrete cosine transformation are input to the watermark embedding device 1107. One DCT coefficient is selected 
from one block, and a bit representing watermark information (to be referred to as a "embedding bit" hereinafter) is 
embedded in the selected DCT coefficient. Note that frequency components in which embedding bits are to be em- 
bedded are randomly selected from components having relatively low frequencies except for DC components. 
[0090] The watermark embedding process is completed by quantizing the DCT coefficients in which embedding bits 
are embedded. The size of a quantization step in this case corresponds to the embedding strength. 
[0091] The following is an example of embedding the bit "0" or "1" for embedding watermark information (to be 
referred to as a "embedding bit" hereinafter). A value q is obtained by quantizing a value s{u 0 v 0 } of a DCT coefficient 
as per: 

q = «s{u 0 v 0 }/h»h (5) 

where s{u 0 v 0 } is the value of the DCT coefficient, h is the embedding strength, and «x» is a maximum integer that 
does not exceed x. 

[0092] A value c that satisfies equation (6) or (7) and is nearest to s{u 0 v 0 } is set as a DCT coefficient after the 

watermark embedding process. 

[0093] If the embedding bit is '0", then 

c = q + ht + q/4 (6) 

[0094] If the embedding bit is "1 then 

c = q + ht + 3q/4 (7) 

where t is a natural number. 

[0095] In the above watermark embedding process, an initial value set in a random generator for generating random 
numbers for specifying embedding components and the value of a quantization step are important. 
[0096] The apparatus in which the processes from the image data compressing process to the watermark embedding 
process are integrated has been described above. In general, the processes from the discrete cosine transformation 
device 11 02 to the memory 1105 areoften integrated into an image compression unit 1101. When, therefore, watermark 
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information is to be embedded in the compressed image data stored in the memory 1105, a watermark embedding 
process can be efficiently implemented by only adding the processed performed between the entropy decoder 1106 
and the entropy coder 1108 in Fig. 10. 

[0097] When an image data compressing process and watermark embedding process are to be continuously per- 
5 formed, compressed image data can be directly input to the entropy decoder 1106 without being input to the memory 
1 105. In addition, the output from the entropy coder 1 108 can be used as an output from the image processing apparatus 
1101 without being input to the memory 1105. 

[0098] If the memory 1 1 05 is used, a watermark embedding process can be performed independently of a compress- 
ing process for image data. For example, therefore, a watermark embedding process can be selectively performed 
io when compressed image data is output from the image processing apparatus 1101 or when image data has not un- 
dergone a compressing process, i.e., the image data x is input. 

[0099] Furthermore, if watermark embedding is not required, the memory 1105 may be controlled to inhibit com- 
pressed image data from being output to the entropy decoder 1106. 

is <Sixth Embodiments 

[0100] An image processing apparatus according to the sixth embodiment of the present invention will be described 
below. The same reference numerals as in the fifth embodiment denote the same parts in the sixth embodiment, and 
a detailed description thereof will be omitted. 
20 [0101] In the watermark embedding method of the fifth embodiment, the DCT coefficient s{u 0 v 0 } in «s{u 0 v 0 }» in 
equation (5) is quantized by using the parameter h. Since this quantization differs from that in a data compressing 
process, the compressed image data must be restored to DCT coefficients by using the entropy decoder 1106 and 
inverse quantizer 1109 in the fifth embodiment. 

[01 02] If quantization used in a watermark embedding process is identical to that used in a data compressing process 
2S or a process is performed in consideration of quantization used in a data compressing process, watermark information 
can be embedded in quantized data without requiring the inverse quantizer 1109. 

[0103] Fig. 11 is a block diagram showing the arrangement of an image processing apparatus that does not require 
the inverse quantizer 1109. 

[0104] If, for example, the parameter h in equation (5) is set to the same value as that of a quantization step used 
30 jn data compression, since the entropy -decoded quantized data is equivalent to «s{u 0 v 0 }/h», the value q can be 
calculated by using this value. Obviously, watermark information can therefore be embedded. 
[0105] In the above case, quantization in a data compressing process is the same as that used in a watermark 
embedding process. If quantization is used in a watermark embedding process in consideration of quantization used 
in a compression process, i.e., the values of the two quantization steps have some relationship, entropy-decoded 
35 quantized data can be converted into quantized data required for a watermark embedding process. If, therefore, quan- 
tization is used in a watermark embedding process in consideration of quantization used in a compressing process, 
watermark information can be embedded in the compressed image data without using the inverse quantizer 1109, as 
shown in Fig. 12B. 

40 <Seventh Embodiment 

[0106] An image processing apparatus according to the seventh embodiment of the present invention will be de- 
scribed below. The same reference numerals as in the fifth embodiment denote the same parts in the seventh embod- 
iment, and a detailed description thereof will be omitted. 

45 [0107] In the fifth and sixth embodiments described above, watermark information is embedded by using the discrete 
cosine transformation. According to the present invention, however, watermark information can be embedded by using 
various image conversion schemes other than the discrete cosine transformation. In JPEG 2000, for example, the 
discrete wavelet transformation is promising as image conversion, and is expected to use overall compression/decom- 
pression procedures indicated by the arrangement shown in Figs. 1 A and 1 B. The present invention is therefore ef- 

50 fective for JPEG 2000 as well. As other image conversion schemes, various conversion schemes, such as the discrete 
Fourier transformation and predictive coding, are available. If these converted data are compressed by quantization 
or entropy coding, the watermark embedding process of the present invention is effective. 

[0108] In addition, various embedding techniques can be used in the watermark embedding device 1107 in a case 
wherein the same image conversion is used in both a data compressing process and a watermark embedding process 
55 as well as in the watermark embedding processes in the fifth and sixth embodiments. 

[0109] In the fifth and sixth embodiments, watermark information is embedded in image data. However, the present 
invention can be applied to a case wherein various data compressing processes are performed for sound data including 
speech data, and watermark information is embedded in the compressed data as well as the case wherein watermark 
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information is embedded in image data. 

iTshJ ™™fI TwJSIT* embeddin9 b,ocks in * e mh and sbcth embodimen,s can be s °>™^ —"fl* 

[0111] As shown in Fig. 13, a switch 1111 may be provided before a quantizer 1103 to selectively outpul each DCT 
coefficienUnput to the quantizer 1103, from one of the outputs of an image converter 1102 and watermark embedding 
device 1107. With th.s arrangement, in the fifth and sixth embodiments, an entropy coder 1104 and quantizer 1103 can 
be shared in an image data compressing process and watermark embedding process. This makes it possible to realize 
an apparatus with higher efficiency. 

[0112] In an image processing apparatus including an image compression block and image decompression block 
as shown in Fig. 1 4, the output from an inverse quantizer 1109 for image decompression is supplied to the watermark 
embedding dev.ce 1107. In addition, since the switch 1111 is provided before the quantizer 1103, not only the pair of 
entropy coder 1104 and quantizer 1103 but also the pair of entropy decoder 1106 and inverse quantizer 1109 can be 
Zr^/r 1 , 3 Wa,ermark embed <*ng process. This makes it possible to realize an apparatus with higher effi- 
c ency. Note that an image inverse converter 1112 performs a process inverse to that performed by the image converter 



1102. 

<Eighth Embodiment 



[0113] An image processing apparatus according to the eighth embodiment of the present invention will be described 
belowThe same reference numerals as in the fifth to seventh embodiments denote the same parts in the eighth 
embodiment, and a detailed description thereof will be omitted. 

[0114] Figs. 15 and 16 are block diagrams each showing the arrangement of an apparatus for extracting watermark 
information from compressed data in which the watermark information is embedded. The arrangement shown in Fiq 
15 corresponds to the watermark embedding apparatus shown in Fig. 12A. The arrangement shown in Fig 16 corre- 
sponds to the watermark embedding apparatus shown in Fig. 1 2B, which requires no inverse quantizer 

n,l!LJ ° f thS appara,uses shown in R 9 S - « and 16 is compressed data in which watermark 

information is embedded. The input compressed data is converted into quantized data by an entropy decoder 501 
corresponding to the entropy coder 1108 used for watermark embedding. In the arrangement shown in Fig 15 the 
obtained quantized data is converted into coefficients dependent on an image conversion method, e g DCT coeffi- 
cients, by an inverse quantizer 502 corresponding to the quantizer 1110 used for watermark embedding The coeffi- 
cients are then input to a watermark extracting device 503. In the arrangement shown in Fig. 16, the quantized data 
is input to a watermark extracting device 503. 

[01 16] The watermark extracting device 503 extracts the embedded watermark information by an extracting process 
corresponding to the watermark embedding process. This watermark extracting process may be a technique corre- 
sponding to the watermark embedding process using the discrete cosine transformation described in the fifth and sixth 
embodiments or the watermark embedding process using various image conversion schemes described in the seventh 
embodiment. 

[01 17] The image processing apparatus according to each of the first to eighth embodiments can be applied to imaqe 
input apparatuses such as digital still cameras and digital video cameras (to be collectively called -digital cameras" 
hereinafter). Assume that the image compressing process in this image input apparatus is provided as hardware Even 
in this case, if a watermark embedding process can be easily implemented by supplying a program to the CPU without 
ado^ng any new hardware. In addition, a watermark embedding process need not always be performed in real time 
unlike a data compressing process, and hence can be executed while data is transferred from an image input apparatus 
such as a digita camera to a personal computer or the image input apparalus executes no image input process This 
makes rt possible to embed watermark information within a short period of time even if the performance of the CPU 
mounted in the image input apparatus is low. 

[0118] As is also obvious, the image processing apparatus according to each of the first to eighth embodiments can 
be applied to an .mage input/output apparatus such as a color facsimile apparatus. In this case, the watermark infor- 

PvSinT. ? ^J^T aPPar8lUS 18 eXtraCt6d by the image ou, P ut a PP ara,us in which the watermark 
extracting function descnbed in the fourth or eighth embodiment is installed. If regulation information (restriction infor- 
ma^on about an output resolution and output size/number of output colors) about an image output is indteated by the 
rorm t a 1S, Wa,ermark in, ° rma,i ° n ' ,he ima9e ° uXpiA a PP ara,us can °e controlled in accordance with the regulation 

[0119] Obviously, the first to eighth embodiments described above can be applied to various systems such as a still 
image transfer system having a compression application such as JPEG and still image processing system 
[0120] As has been described above, according to the first to eighth embodiments described above watermark 
information can be embedded in compressed data and embedded watermark information can be extracted by adding 
a simple processing means to an image processing apparatus having an image compressing function 
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[01 21] The present invention can be applied to a system constituted by a plurality of devices (e.g., a host computer, 
interface, reader, printer, etc.) or to an apparatus comprising a single device (e.g., a copier or facsimile machine, etc.). 
[0122] Furthermore, it goes without saying that the invention is applicable also to a case where the object of the 
invention is attained by supplying a storage medium storing the program codes of the software for performing the 
5 functions of the foregoing embodiment to a system or an apparatus, reading the program codes with a computer (e. 
g., a CPU or MPU) of the system or apparatus from the storage medium, and then executing the program codes. 
[01 23] In this case, the program codes read from the storage medium implement the novel functions of the invention, 
and the storage medium storing the program codes constitutes the invention. 

[0124] Further, the storage medium, such as a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, 

io CD-R, magnetic tape, non-volatile type memory card or ROM can be used to provide the program codes. 

[01 25] Furthermore, besides the case where the aforesaid functions according to the embodiment are implemented 
by executing the program codes read by a computer, it goes without saying that the present invention covers a case 
where an operating system or the like running on the computer performs a part of or the entire process in accordance 
with the designation of program codes and implements the functions according to the embodiments. 

is [0126] It goes without saying that the present invention further covers a case where, after the program codes read 
from the storage medium are written in a function expansion board inserted into the computer or in a memory provided 
in a function expansion unit connected to the computer, a CPU or the like contained in the function expansion board 
or function expansion unit performs a part of or the entire process in accordance with the designation of program codes 
and implements the function o1 the above embodiment. 

20 [01 27] Thus, in accordance with the present invention, as described above, amplitude altering magnifications which 
differ for voiced and unvoiced sounds are used to perform multiplication when the power of synthesized speech is 
controlled. This makes possible speech synthesis in which noise-like abnormal sounds are produced in unvoiced sound. 
[0128] As many apparently widely different embodiments of the present invention can be made without departing 
from the spirit and scope thereof, it is to be understood that the invention is not limited to the specific embodiments 

2S thereof except as defined in the appended claims. 



Claims 

30 1. An image processing apparatus comprising: 

inputting means for inputting image data which has been coded in accordance with a coding table; and 
embedding means for embedding added information in the image data by modifying the input image data 
based on the coding table. 

35 

2. The apparatus according to claim 1 , wherein the added information is electric watermark information. 

3. The apparatus according to claim 1 , wherein the input image data has been coded by using discrete cosine trans- 
formation. 

40 

4. The apparatus according to claim 1, wherein the input image data has been coded by using discrete wavelet 
transformation. 

5. An image processing method comprising the steps of: 

45 

inputting image data which has been coded in accordance with a coding table; and 

embedding added information in the image data by modifying the input image data based on the coding table. 

6. A computer program product comprising a computer readable medium having computer program code, for an 
so image processing method, said product comprising: 

input process procedure code for inputting image data which has been coded in accordance with a coding 
table; and 

embed process procedure code for embedding added information in the image data by modifying the input 
55 image data based on the coding table. 

7. An information processing apparatus comprising: 
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inputting means for inputting image data which has been quantized in accordance with a quantizing table; and 
embedding means for embedding added information in the image data by modifying the input image data 
based on the quantizing table. 

8. The apparatus according to claim 7, wherein the added information is electric watermark information. 

9. The apparatus according to claim 7, wherein the input image data has been subjected to discrete cosine transfor- 
mation. 

10. The apparatus according to claim 7, wherein the input image data has been subjected to discrete wavelet trans- 
formation. 

11. An information processing method comprising the steps of: 

inputting image data which has been quantized in accordance with a quantizing table; and 

embedding added information in the image data by modifying the input image data based on the auantizina 

table. H y 

12. A computer program product comprising a computer readable medium having computer program code, for an 
information processing method, said product comprising: 

input process procedure code for inputting image data which has been quantized in accordance with a quan- 
tizing table; and 

embed process procedure code for embedding added information in the image data by modifying the input 
image data based on the quantizing table. 

13. An information processing apparatus comprising: 

decoding means for decoding and inversely quantizing coded and quantized data; 

embedding means for embedding added information in the data output by said decoding means; and 

re-coding means for quantizing and coding the data output by said embedding means. 

14. The apparatus according to claim 13, wherein the added information is electric watermark information. 

15. The apparatus according to claim 1 3, wherein said decoding means inputs the coded and quantized data repre- 
senting an image has been subjected to discrete cosine transformation or discrete wavelet transformation. 

.16. The apparatus according to claim 1 3, wherein said decoding means determines whether the coded and quantized 
data is processed or not in a predetermined unit. 

17. An information processing method comprising the steps of: 

decoding and inversely quantizing coded and quantized data; 

embedding added information in the data output in said decoding step; and 

re-quantizing and re-coding the data output in said embedding step. 

18. A computer program product comprising a computer readable medium having computer program code, for an 
information processing method, said product comprising: 

decoding process procedure code for decoding and inversely quantizing coded and quantized data; 
embedding process procedure code for embedding added information in the data output by said decoding 
process; and 

re-coding process procedure code for quantizing and coding the data output by said embedding process. 

19. An information processing apparatus comprising: 

decoding means for decoding coded data; 

embedding means for embedding added information in the data output by said decoding means; and 
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10 



re-coding means for coding the data output by said embedding means. 

20. The apparatus according to claim 19, wherein the added information is electric watermark information. 

21. The apparatus according to claim 19, wherein said decoding means inputs the coded data representing an image 
has been subjected to discrete cosine transformation or discrete wavelet transformation. 

22. The apparatus according to claim 1 9, wherein said decoding means determines whether the coded data is decoded 
or not in a predetermined unit 

23. An information processing method comprising the steps of: 



decoding coded data; 

embedding added information in the data output in said decoding step; and 
is re-coding the data output by said embedding step. 

24. A computer program product comprising a computer readable medium having computer program code, for an 
information processing method, said product comprising: 

20 decoding process procedure code for decoding coded data; 

embedding process procedure code for embedding added information in the data output by said decoding 
process; and 

re-coding process procedure code for coding the data output by said embedding process. 

2$ 25. An information processing apparatus comprising: 

decoding means for decoding and inversely quantizing coded and quantized data; and 

extracting means for extracting added information embedded in the data which is output by said decoding 

means. 

30 

26. The apparatus according to claim 25, wherein the added information is electric watermark information. 

27. The apparatus according to claim 25, wherein said decoding means inputs the coded data representing an image 
has been subjected to discrete cosine transformation or discrete wavelet transformation. 

28. The apparatus according to claim 25, wherein said decoding means determines whether the coded data is decoded 
or not in a predetermined unit. 

29. An information processing method comprising the steps of: 

decoding and inversely quantizing coded and quantized data; and 

extracting added information embedded in the data which is output in said decoding step. 

30. A computer program product comprising a computer readable medium having computer program code, for an 
45 information processing method, said product comprising: 

decoding process procedure code for decoding and inversely quantizing coded and quantized data; and 
extracting process procedure code for extracting added information embedded in the data which is output by 
said decoding process. 



35 



40 



so 



31. An image processing apparatus comprising: 



generating means for generating image data; 

compressing means for compressing the image data generated by said generating means; and 
55 embedding means for embedding added information in the image data compressed by said compressing 

means, 

wherein said compressing means and said embedding means can be executed in independently 
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32. The apparatus according to claim 31, wherein the added information is electric watermark information. 

33. The apparatus according to claim 31, wherein said embedding means is executed when said image processing 
means outputs data or said generating means does not operate. 

34. An image processing method comprising the steps of: 

generating image data; 

compressing the image data generated in said generating step; 

embedding added information in the image data compressed in said compressing step; and 
controlling operations of said compressing and embedding steps in independently. 

35. A computer program product comprising a computer readable medium having computer program code for an 
image processing method, said product comprising: 

generating process procedure code for generating image data; 

compressing process procedure code for compressing the image data generated by said generating process- 
embedding process procedure code for embedding added information in the image data compressed by said 
compressing process; and 

controlling process procedure code for controlling operations of said compressing and embedding processes 
in independently. 

36. A signal conveying instructions for causing a programmable processing apparatus to become operable to perform 
a method as set out in at least one of claims 5, 11 , 17, 23, 29 and 34. 
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FIG. 1A 
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FIG. 2A 
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FIG. 4 
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FIG. 5 
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FIG. 6 
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FIG. 7 
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FIG. 9A 
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